Fix K8s executor pod_override stringified without cncf provider#67895
Merged
jscheffl merged 2 commits intoJun 7, 2026
Conversation
Airflow 3 deployments using the Kubernetes executor with the `kubernetes` package installed but without `apache-airflow-providers-cncf-kubernetes` had their `executor_config` `pod_override` V1Pod silently converted to a string during Dag serialization. The stringified pod was persisted to `serialized_dag` and `task_instance.executor_config`, so affected tasks stayed in `queued` and never ran. `_has_kubernetes()` hardcoded an import of the cncf provider's `PodGenerator` for V1Pod ser/deser. When only an alternative executor package is installed, that import failed, `_has_kubernetes()` cached False, and serialization fell back to `str(pod)`. Airflow 2 had an in-core `PodGenerator` fallback for this case that was removed ahead of the 3.0 release. Serialize and deserialize V1Pod objects directly through `kubernetes.client.ApiClient`, which is all `PodGenerator.serialize_pod` and `deserialize_model_dict` wrap anyway. V1Pod ser/deser now works with the cncf provider, an alternative executor package, or the bare `kubernetes` package installed. The backcompat unpickle path in `ensure_pod_is_valid_after_unpickling` is decoupled the same way.
ephraimbuddy
added
the
backport-to-v3-2-test
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes an Airflow 3 regression where Kubernetes executor executor_config["pod_override"] (kubernetes.client.models.V1Pod) could be silently stringified during DAG serialization when kubernetes is installed but apache-airflow-providers-cncf-kubernetes is not, causing queued tasks to never run. The change removes the hard dependency on the cncf provider’s PodGenerator by serializing/deserializing V1Pod directly via kubernetes.client.api_client.ApiClient.
Changes:
- Replace cncf-provider
PodGeneratorusage with directApiClient-basedV1Podserialization/deserialization inBaseSerialization. - Decouple the backcompat unpickle repair path (
ensure_pod_is_valid_after_unpickling) from the cncf provider. - Add a regression test that simulates the cncf provider being unimportable and asserts
V1Podstill round-trips throughBaseSerialization.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| airflow-core/src/airflow/serialization/serialized_objects.py | Switch POD ser/deser from provider PodGenerator to direct kubernetes ApiClient, and update the missing-dependency error message. |
| airflow-core/src/airflow/utils/sqlalchemy.py | Update the unpickling repair path to deserialize V1Pod via ApiClient rather than the cncf provider. |
| airflow-core/tests/unit/serialization/test_serialized_objects.py | Update existing V1Pod import/patch test and add regression coverage for “no cncf provider installed”. |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
jscheffl
approved these changes
Jun 7, 2026
Contributor
Backport failed to create: v3-2-test. View the failure log Run detailsNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
You can attempt to backport this manually by running: cherry_picker 4677e1e v3-2-testThis should apply the commit to the v3-2-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continueIf you don't have cherry-picker installed, see the installation guide. |
jscheffl
added a commit
that referenced
this pull request
Jun 7, 2026
…ovider (#67895) (#68157) * Fix Kubernetes executor pod_override stringified without cncf provider Airflow 3 deployments using the Kubernetes executor with the `kubernetes` package installed but without `apache-airflow-providers-cncf-kubernetes` had their `executor_config` `pod_override` V1Pod silently converted to a string during Dag serialization. The stringified pod was persisted to `serialized_dag` and `task_instance.executor_config`, so affected tasks stayed in `queued` and never ran. `_has_kubernetes()` hardcoded an import of the cncf provider's `PodGenerator` for V1Pod ser/deser. When only an alternative executor package is installed, that import failed, `_has_kubernetes()` cached False, and serialization fell back to `str(pod)`. Airflow 2 had an in-core `PodGenerator` fallback for this case that was removed ahead of the 3.0 release. Serialize and deserialize V1Pod objects directly through `kubernetes.client.ApiClient`, which is all `PodGenerator.serialize_pod` and `deserialize_model_dict` wrap anyway. V1Pod ser/deser now works with the cncf provider, an alternative executor package, or the bare `kubernetes` package installed. The backcompat unpickle path in `ensure_pod_is_valid_after_unpickling` is decoupled the same way. * Apply suggestions from code review --------- (cherry picked from commit 4677e1e) Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Airflow 3 deployments that run the Kubernetes executor with the
kubernetespackage installed but without
apache-airflow-providers-cncf-kuberneteshad their
executor_configpod_overrideV1Podsilently converted to astring during Dag serialization. The stringified pod was persisted to
serialized_dag._dataandtask_instance.executor_config, so affected tasksstayed in
queuedand never ran. (Airflow 2 handled this case via an in-corePodGeneratorfallback that was removed ahead of the 3.0 release.)Root cause
_has_kubernetes()inserialized_objects.pyhardcoded an import of the cncfprovider's
PodGeneratorforV1Podser/deser. When that provider isn'tinstalled the import fails,
_has_kubernetes()cachesFalse, theV1Podserialization branch short-circuits, and
default_serialization()falls backto
str(var).Fix
Serialize/deserialize
V1Podobjects directly throughkubernetes.client.ApiClient— which is allPodGenerator.serialize_podandPodGenerator.deserialize_model_dictwrap anyway — so airflow-core no longerdepends on any specific Kubernetes-executor provider.
V1Podser/deser nowworks with the cncf provider, an alternative executor package, or just the bare
kubernetespackage installed. The backcompat unpickle path inensure_pod_is_valid_after_unpicklingis decoupled the same way.Tests
Added a regression test that makes
apache-airflow-providers-cncf-kubernetesunimportable and asserts
V1Podstill round-trips throughBaseSerialization. The test blocks the...pod_generatorsubmodule directly(not just the parent package) so it can't be fooled by import ordering when the
module is already cached by a sibling test. Existing K8s serialization tests
(
test_serialize_v1pod_attempts_import_before_serializing, theexecutor_config_podfixtures,TestExecutorConfigType) continue to pass —serialization output is byte-identical to the previous cncf-based path.
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.8) following the guidelines